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Abstract — Loss tomography has received considerable atten- 
tion in recent years and a number of estimators based on 
maximum lilcelihood (ML) or Bayesian principles have been 
proposed. Almost all of the estimators are devoted to the tree 
topology despite the general topology is more common in practice. 
There has been few likelihood function devoted to the general 
topology, not to mention the estimator. To overcome this, two 
sets of sufficient statistics for the tree and general topologies, 
respectively, are proposed in this paper. Using the statistics, two 
likelihood functions, one for a topology, are proposed here and 
subsequently two likelihood equations for the general topology, 
one is link-based and the other is path-based, are obtained. 
In addition, a dependence between subtrees in terms of their 
estimates is identified for the general topology and a divide-and- 
conquer strategy is proposed to deal with the dependence, which 
divides a general network into two types of independent trees. 
Further, two algorithms, one for a type of the independent trees, 
are proposed to estimate the loss rates of each type. 

Index Terms — Decomposition, General topology, Likelihood 
Equations, Loss tomography. Tree topology. 



I. Introduction 

Network characteristics, such as loss rate, delay, available 
bandwidth, and their distributions, are critical to various 
network operations and also important to network research, 
e.g modeling. Because of these, considerable attention has 
been given to network measurement, in particular to large 
networks. However, due to various reasons, e.g. security, 
commercial interests and administrative boundary, some of 
the characteristics cannot be obtained directly from a large 
network. To overcome the limitations, network tomography 
is proposed in |[T|, where the author suggests to use end- 
to-end measurement and statistical inference to estimate the 
characteristics of a large network. The characteristics that have 
been estimated in this manner include link-level loss rates 
delay distribution | J|, [41, I'S'l, (6\, {T\, and loss pattern (S]. In 
this paper, our attention is focused on the loss rate inference, 
which can be easily extended to estimate loss pattern. 

In an active approach, a number of sources attached to a 
network send probes to the receivers attached to the other 
side of the network, where the paths from the sources to 
the receivers cover the network of interest. To create the 
correlation required for estimation, probes need to be sent 
in a coordinated manner to the receivers, where a multicast 
scheme is often used to achieve this. Then, the arrivals, 
arrival orders and arrival time of the probes are used by 
an estimator to infer the characteristics of interest. If the 
probes sent in an experiment are far apart and the traffic is 
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Statistically stable, the observations obtained at receivers are 
considered independent identical distributed (i.i.d.) and the 
likelihood function of the observations is the product of the 
individual ones. To determine the parameter(s) embedded into 
a likelihood function, ML or Bayesian principles are often 
used in the determination. No matter which principle is used 
in estimation, the sufficient statistics from observations are 
crucial for an estimator Unfortunately, this issue has been 
overlooked in the past that leads some of the estimators 
proposed so far to use a part of the available information. 

Apart from statistics, most of works in loss tomography 
have been focused on the tree topology although a general 
topology is more common in practice. The tree topology as 
named has a single source attached to the root of a multicast 
tree to send probes to the receivers attached to the leaf nodes 
of the multicast tree, where all of the receivers have a common 
view of the probing process. A network of the general topology 
is different from the tree one that is composed of a number 
of trees and some parts of the trees are intersected with each 
other. Thus, the receivers, nodes, and links in an intersected 
subtree can receive probes from multiple sources and observe 
a number of probing processes. Because of the difference, 
an estimator for the general topology, in particular for the 
intersected areas, must consider the probes sent by different 
sources. In addition, the estimator must consider the possible 
dependence between intersected areas and other areas. The 
differences make the estimators or algorithms developed for 
the tree topology incapable of the general topology. This paper 
aims at overcoming the shortage that will address the statistics, 
the correlation and the dependence raised above. In addition, it 
provides a number of ML estimators for the general topology. 

A. Contribution and Paper Organization 

As stated, there has been a lack of ML estimators in loss 
tomography for the tree topology and there has been a lack of 
both analytical results and estimators for the general topology. 
To fill the gaps, we in this paper present the results obtained 
recently that partially solve the problems, which, comparing to 
lUl- ifToi . have two-fold contributions to loss tomography, one 
for a topology. For the tree topology, there are 2 contributions: 
1) A set of the minimal sufficient statistics is introduced 
and used to rewrite the widely used likelihood function 
into a different format. From the likelihood function, a 
set of likelihood equations is derived that shows the loss 
rate of a link depends on the loss rates of its ancestors 
and descendants, where the polynomial proposed in O 
is a special case of the likelihood equations. 
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2) The solution space is proved to be concave if a Bernoulli 
model is assumed to describe the loss behavior of a link. 
The finding ensures that the estimates obtained by an 
iterative procedure, e.g. EM, are MLEs. 

For the general topology, there are four contributions: 

1) The analytical results obtained from the tree topology 
are extended to the general topology, and a direct 
expression of the MLE is derived for the loss rate of 
a link. The direct expression has a similar structure as 
that of the tree topology. 

2) By proving the direct expression leading to an MLE, we 
also prove the solution space is concave and the estimate 
obtained by an iterative algorithm, such as the EM, the 
fixed point and MCMC, is an MLE. 

3) Apart from the link-based estimator, a path-based esti- 
mator is proposed that not only generalizes the results 
presented in ||9l, but also points out the specialty of 
the general topology. The specialty rests on the demand 
of consistency in estimation that leads to an order in 
estimation. 

4) A divide-and-conquer strategy is proposed to impose the 
order raised above that decomposes a general network 
into two type of independent trees; one is for those 
without any intersection with others, the other is for 
those with intersection(s). Further, two estimators are 
proposed, one for each of the above. 

Apart from the above, a practical issue that has been over- 
looked by previous works is raised in this paper, which is 
about the number of probes needed to capture the loss rate 
of a link. In contrast to previous works, the focus here is not 
only on the number of probes that need to be sent but also on 
the consistency between the background traffic and the model 
used by an estimator Only if the model used to develop the 
estimator is consistent with that of the background traffic, is 
the accuracy of the estimator guaranteed. If not, however, there 
is no guarantee that an estimate obtained from more samples is 
better than that obtained from less ones. Our simulation study 
shows that for some background traffic, 5% variance can be 
easily achieved with a few hundreds of samples. Nevertheless, 
for some other traffic, 10% variance is hardly achieved even 
with a few thousands of samples. In this case, increasing the 
percentage of the probes in traffic can improve estimation 
accuracy, i.e. sacrificing efficiency for accuracy. Despite this, 
the estimate obtained may not be the true one since the probes 
sent can intervene the background traffic. 

The rest of the paper is organized as follows. In Section 
Hn we present the related works and the notations used in this 
paper In addition, a set of sufficient statistics are introduced 
in this section. There are four parts in Section HIl] a) a 
likelihood function of the tree topology is created from the 
set of sufficient statistics; b) the likelihood equation from 
the likelihood function is derived; c) the likelihood equation 
derived in b) and that presented previously are compared 
and the difference between them are identified; and d) the 
statistics introduced in Section|lI]are proved to be the complete 
minimal sufficient ones. We then extend the results from the 
tree topology to the general topology in the rest of the paper. 



Section |IV] contains the extended notation and the set of 
sufficient statistics. Then, the link-based and the path-based 
likelihood equations are presented in Section W\ Section IVT] is 
devoted to the solution of the likelihood equations proposed in 
SectionlV] Statistical properties of the estimators are discussed 
in Section IVIII Section IVIIII presents simulation results. The 
last section is devoted to concluding remark. 

II. Related Works and Problem Formulation 

A. Related Works 

Multicast Inference of Network Characters (MINC) is the 
pioneer of using multicast probes to create correlated obser- 
vations, where a Bernoulli model is used to model the loss 
behaviors of a link. Using this model, the authors of |9 | derive 
an MLE for the pass rate of a path connecting the source to 
a node. The MLE is expressed in a polynomial that is one 
degree less than the number of descendants of the node [9|, 
ifTTll . lfT2l . To ease the concern of using numeric method to 
solve a higher degree polynomial (> 5), the authors of f\3\ 
propose an explicit estimator Although the estimator has the 
same asymptotic variance as that of the MLE to first order, it 
is not an MLE and there is a noticeable difference between the 
estimate obtained by the estimator and an MLE if n < oo. The 
two estimators are dedicated to the tree topology. Later, Bu 
et. al. attempted to extend the strategy to the general topology 
without success in [2j. The authors then resorted on iterative 
procedures, i.e. the EM, to approximate the maximum of the 
likelihood function. In addition, a minimum variance weighted 
average (MVWA) method is used for comparison with the 
EM one. The problem of MVWA is the bias of its estimate 
when the sample size is small |14|. The experimental results 
presented in [2| confirm this, where the EM outperforms 
the MVWA when the sample size is small. Nevertheless, the 
estimates obtained by the EM algorithm could not be verified 
to be the MLE since there was no proof that the search space is 
concave. In addition, iterative procedures, e.g. EM algorithm, 
have their interionic weakness as previously stated in |15|. 
Considering the unavailability of multicast in some networks, 
Harfoush et al. and Coates et al. independently proposed the 
use of the unicast-based multicast to send probes to receivers 
GSl, iflTl . where Coates et al. also suggested the use of the EM 
algorithm to estimate link-level loss rates. Apart from those, 
Rabbat et al. in 1 18| consider network tomography on general 
networks and found a general network comprised of multiple 
sources and multiple receivers can be decomposed into a num- 
ber of 2 by 2 components. The authors further proposed the 
use of the generalized likelihood ratio test to identify network 
topology. To improve the scalability of an estimator, Zhu and 
Geng propose a bottom up estimator for the tree topology in 
fr9l, which later is found to be topology independent ifTol . 
The estimator adopts a step by step approach to estimate the 
loss rate of a link, one at a time and from bottom up. At each 
step the estimator uses a formula to compute the loss rate of a 
link. Despite the effectiveness, scalability, and extensibility to 
the general topology, the estimate obtained by the estimator is 
not the MLE as the one proposed in 1 13 1 because the statistics 
used by the two estimators are not sufficient ones. 



B. Notation 

Let T = (y, E, 9) donate the multicast tree, where V = 
{vo,vi, ...Vm} is the set of nodes representing routers and 
switches of a network; E ~ {ei, Cm} is the set of directed 
links connecting the nodes of V, the two nodes connected by 
a link are called the parent node and the child node of the link, 
where the parent forwards probes received from its parent to 
the child; and 9 — {6i, 9m} is the set of parameters to be 
estimated, one for a link. The multicast tree used to deliver 
probes to receivers is slightly different from an ordinary one 
at its root that has only a single descendant. Figure [1] presents 
an example of the multicast tree. 

To distinguish a link from another, each link is assigned a 
unique number from 1 to m; similarly, each node also has a 
unique number from to m, where link i is used to connect 
node i's parent node to node i. The numbers assigned to the 
nodes are from small to big along the path of the Breadth-first 
traversal. The source attached to node sends probes to the 
receivers attached to the leaf nodes. R is used to denote the 
receivers attached to T. In contrast to [9 1 and [13] that use node 
as reference in discussions, we use link instead because there 
is no one-to-one correspondence between nodes and links in 
the general topology. As a hierarchical structure, each link in 
a tree except the root and leaves has a parent and a number of 
descendants. If fi{i), simply f{i) later, is used to denote the 
parent link of link i and fi{i) to denote the ancestor that is I 
hops away from link i in the path to the root, we have fk{i) — 
/(/fe_i(i)). In addition, let a{i) = {f{i),f2{i),--Jk{i)}, 
where fk{i) = 1, denote the ancestors of link i. Further, di 
denotes the descendants of link i and \di\ denotes the number 
of descendants in di. To distinguish subtrees, each multicast 
subtree is named by the number assigned to its root link, 
where T{i) = {V{i), E{i),9{i)},i e {l,--,m} denotes the 
multicast subtree rooted at node f{i), where V{i), E{i) and 
d{i) are the nodes, links and parameters of T{i). Note that 
T(i), i ^ R, is rooted at node f{i) that uses link i to connects 
subtree i. The group of receivers attached to T{i) is denoted 
by R{i). If n probes are dispatched from the source, each 
probe i — 1, n gives rise of an independent realization 
of the loss process X. Let XI denote the state of link k for 
probe i, X], = 1, fc € i? if probe i passes link fc; otherwise 
XI, —{). ^ — k E R comprise the observations 

of an experiment. In addition, ftj is the observations of R{j) 
and (y^^y'=i^2,...,n^ k E E is the state of link k inferred from 
observation i. = 1 if at least one of i?(fc) receives probe 
i, otherwise Yj_f — 0. 

C. Internal State and Internal View 

Given the loss model of a link, statistical inference is 
used to determine the unknown parameters of the likelihood 
function. If the ML principle is applied, the task is frequently 
described as the maximization of the following log-likelihood 
function ||9l : 

argmaxL(0) = argmax > n(x)los:P(x:9) (1) 

where <d — [0, 1)™ is the value space of parameter 9, n{x) is 
the number of observation x,x E ft, and P{x; 9) is the joint 
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Fig. 1. Network structure 

probability of observation x with parameter 9. However, it is 
hard, if not impossible, to derive the likelihood equation from 
a log-likelihood function in the form of ([Til. 

Instead of using the log-likelihood function as ([T]|, we 
consider to rewrite it in a different form. Let Pn{9) denote 
the likelihood function. Under the i.i.d. assumption of probes, 
we have 

n 

and under the Bernoulli assumption of losses at a link, we 
have 

p(x\ 9)^^{\- 9,) n (^fc + (1 - - m 

j£El k£E2 

for X\ where El denotes the set of links that have Yl- = 1; 
E2 denotes the set of links whose observation of the probe 
cannot be confirmed but their parents' are confirmed, i.e. 

= 1 and r,! - O; and /3, = P{\/reRi^)^r = l\X., = 
1;9) is the pass rate of subtree i. If we do not distinguish 
the difference between El and E2 and let Pij{9j) denote the 
probability of link j for X^, we have 

n 

Pn{e)^l[l[p^A^,) 

i=ijeE 

With a finite n and \E\, the order of the products is com- 
mutable. To swap the order of the two products, we have 

n 

Having a likelihood function as above, we need to have the 
state of each link for each probe, which results in two concepts 
called internal state and internal view. The formal definitions 
of them are as follows. 

1) Internal State: Given X^ , one is able to obtain X-j^, k E 
R{i) and then we can have 

Y!' = max j E {1, ..,n} 

k£R{i) 

called the internal state of link i for probe j. If Y/ = 1, 
probe j for sure passes link i. Further, considering the values 
of Y^^ and Yjf^^y *, /(*) G E together, we have three possible 
combinations: 

1) Y/ = Yj^^s^ ~ 1, this also means that probe j passes 
link i; or 
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2) Y^-' = and Yj^^^ — 1, this means that probe j reaches 
node f{i) and then become uncertain in T{i), i.e. it is 
not sure whether the probe is lost on Unk i or lost in the 
subtrees rooted at node i; or 

3) = y^(j-) = 0, this means the probe becomes uncertain 
at one of a{i) and the uncertainty is transferred from the 
ancestor to node i. 

For link i, if the first occurs, we need to have (1 — 6';) in the 
likelihood function for this probe; if the second one occurs, we 
need to have {9i + [1 — 9i){\~ Pi)) in the likelihood function; 
and if the last one occurs, we need to have 1 in the likelihood 
function for this probe since P{Y^ = 0\Yj^^^ = 0) = 1. 
Because the likelihood function is in a product form, we only 
need to consider the first two in the likelihood function. 

2) Internal View: Accumulating the states of each link in 
an experiment, and let 



Thus, 



L{Pn{e)) = ^logP(X^0) 

fe=l 



fc=l ieE 

{Yj^^-Yi)\og{e.. + {i-e,){i-p,))] 

^K(l)log(l-&,) + 



ni(O)log((?, + (l-0,)(l-ft))]- 
The theorem follows. ■ 

If the derivative of (|3]l can be obtained and expressed 
explicitly, the likehhood equation of the tree topology becomes 
available. 
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we have the number of probes passing link i confirmed from 
il, i.e. at least probes pass link i in the experiment. In 

addition, let 

ni{0) = ?T./(i)(l) - ni{l) 

denote the number of probes that become uncertain in T{i). 
rii(l) and ni(0) are called the internal view of link i. We will 
prove that ni{l),i E E is a set of sufficient statistics later in 
the paper. Using ni(l), i e £', we are able to have a likelihood 
function of Q, in the format of (|2]i. 



B. Likelihood Equations and Solution 

Differentiating (|3]l with respect to (wrt.) each parameter 
and letting the derivatives be 0, we have a set of likelihood 
equations as: 



dL{Pn{Q)) _ ndl) 

= 0, 



1-9, + {I ~ 9i){l - 13,) 



+ 



n,(o)n,g...[(i-'>,) ,,^(;:,^^'(';_^,) i 



i = 1, ••, m. 



III. New Likelihood Function and Solution 



Reorganizing it, we have 



A. New Likelihood Function 

Given ni{l) and ni{0),i £ E, we then have the following 
theorem to write the log-likelihood function. 

Theorem 1. Given ni{l), ni(0), the log-likelihood function 
of an experiment on the tree topology can be written as 
follows: 

L{Pn{9)) = Y.[n,{l)\og{l-9,) + 

n,{0)\og{9, + {l-9,)il-P,))] (3) 



= < 



n,il) 
n 



i = 1 



n,il) 



nf(,\(l)+imp(f(i)) , , 

Pi 

ni{0) + imp{f{i)) 
. nf{i){l) + imp{J{i)y 



(4) 



i e L 



where L denotes the set of links connecting R. It is clear that 
(|4|i can be solved by approximation if (3i is a function of 9i, 
where 



where 9i and (1 — /3i),i€{l,--, m} are the loss rate of link 
i and the loss rate of subtree i, respectively. 

Proof: Based on the definition of «;(!) and ni{0), we 

have 



P{X^-9) = n [{l-0,f'{9, + {l-9,){l-p,)) 



imp{f{i)) = ^ 



^- PfU) 

nk{0) * pai{k) Uieaiz) 7 

l>k V 



(5) 



is the estimated number of nj{0),j £ a{i) that reaches node 
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f{i) before being lost in T{i), where 

^^ = ^^. + (1-^0(1 -ft) 



pai(k) 



jedi 

n (1-^')- 



l>k 



(6) 



Each term in the summation of Q is for an ancestor j,j G 
a{i) and represents the portion of nj{0) that reaches Unk i, 
where pai{k) is the pass rate of the path from node fk{i) to 

node fii); and ^'^^^^ is the estimate of the number of probes 



reaching node fk(i)- 

C. Similarity and Difference from Previous 

Let /3i = 1, e L, the three formulae of (|4) become 
one as: 

7(a(«)) 



= 1 - 



ft 



(7) 



where 7(0(1)) is the pass rate of T{i), its empirical pass rate 

is equal to 7(0(1)) = As stated, if ft 

is a function of 9i, 9i can be obtained from |4] In fact, the 
expression of ft depends on the observation of R{i). If the 
observation satisfies 

n 

{(E A^^')>oi^j'^'^2'^'\{^'H»' 

1=1 kej 

where 2"^' stands for the set of all subsets of di and {e} denotes 
all single element set, we have. 



jedi ' 



(9) 



If knowing 7(0(1)), we have ft = — , and then (|9]l turns 

1 — Oi 

to a polynomial as that derived in [9], i.e. 



i/.(A.,7)=i-^- n(i-it^=o 



(10) 



Note that (fTOl i is not equivalent to (01 since (|4| is not restricted 
to (HJ. If (HI) does not hold, ft can not be expressed by (|9]l. 
For instance, if , e di has no intersection with others, 
we need to have a different f3i rather than (|9) in (|4). There 
are various /3iS, depending on observations. For those who are 
interested in the detail, please refer to [|20 j . 

D. Proof of Sufficient Statistics 

There are a number of approaches to prove (0) is the 
MLE. One of them is to prove the equivalence between (|4]i 
and ( [Tol l that has been achieved in the previous section. An 
alternative is to prove the internal view proposed in this paper 
is the complete minimal sufficient statistics. To prove this, we 
need to prove that Q belongs to the exponential families. 
That is obvious since (3) follows a Bernoulli distribution. 



Alternatively, according to the following definition presented 
in II22II. 



Definition 1. Let x = {Xi, ....,X„} be a random sample, 
governed by the probability mass function f{x\Ak). The 
statistic T{x) is sufficient for Ak if the conditional distribution 
of X, given T(x) = t, is independent of A^. 

we have theorem |2] that confirms the internal views are 
sufficient statistics. 

Tlieorem 2. ni{l),i £V is a set of sufficient statistics. 

Proof Given the loss process yields the Bernoulli model 
and T{x) — nfc(l), ^ shows the likelihood function is as 
follows: 

/^,(a:)=A;''=«(l-Afeft)"-"'=«. 

The distribution of nfc(l) in n independent Bernoulli trials is 
a binomial as 



(1) 

Then, the conditional distribution can be written as 

/Afc|nj,(l)(a;) = 



1 



(11) 



According to definition [T] {ni(l), • • • ,n„i{l)} are sufficient 
statistics. Since /a^ (x) is of the standard exponential family, 
the statistics are minimal complete sufficient ones. ■ 

IV. Loss Rate Analysis for General Networks 

A. Goals and Background 

As stated, using multiple trees to cover a general network 
makes estimation harder than that in the tree topology. Because 
of this, no likelihood equation has yet been proposed for 
the general topology. Without a likelihood equation, it is 
impossible to know the shape of the search space of an iterative 
algorithm, e.g. the EM algorithm. Subsequently, there is no 
guarantee that the estimate obtained by an iterative method 
is the MLE. To overcome this, two likelihood equations for 
the general topology are presented in the next section: one 
is a link-based estimator and the other is a path-based one. 
The former has a similar structure as (0) that clearly shows 
the independency of the probes sent by multiple sources 
on the estimate of a shared link; the latter defies previous 
approaches by considering the number of probes reaching 
a node from multiple sources. Subsequently, the path-based 
likelihood equation is obtained and the solution space of the 
equation is proved to be strictly concave. Thus, there is a 
unique solution to the likelihood equation that is the MLE. 

B. Extended Notation 

Let J\f denote a general network consisting of k trees, where 
S — {s^ , s'^ , ■ ■ , s'^} denote the sources attached to the trees. 
In addition, V and E denote the nodes and links of J\f. Each 
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node is assigned a unique number, so is a link. \V\ and \E\ 
denote the number of nodes and the number of Hnk of the 
network, respectively. Each of the multicast trees is named 
after the number assigned to its root link, where T{i),i £ E 
denotes the subtree with link i as its root link. Further, let 
R{s),s e S denote the receivers attached to the multicast tree 
rooted at s and let Rs{i), i £ E denotes the receivers attached 
to the multicast subtree rooted at link i. The largest intersection 
between two or more trees is called the intersection of the 
trees and the root of an intersection is called the joint node 
of the intersection. An intersection is an ordinary tree and 
named after its root node. Let J denote all joint nodes and 
let S{j),j E V \ S denote the sources that send probes to 
node j. Since each node can have more than one parents in a 
general network, f{{i), simply later, is used to denote 

the parent of node i on the way to source s and /f (i) to 
denote the ancestor that is I hops away from node i in the 
path to source s. Recursively, we have = f^ifk-ii"^))- 

Further, let a(s,i) = (i), /| W, /I where = 
s, denote the ancestors of node i in the path to s. Let a{i) — 
{a{s,i),s e S{i)} denote all the ancestors of node i. 

If n^,s E S denotes the number of probes sent by source 
s, each probe o = 1,....,ti^ gives rise of an independent 
realization X''{o) of the probe process X. As the tree topology, 
X|(o) = l,fc e £' if probe o sent by source s passing 
link fc; otherwise X^{o) = 0. Further, il — {fls,s € S}, 
ilg = (X''(o)), o = 1, 2, n*, comprises the data set of 
estimation. As the tree, let («), i = 1, 2, n*, k £ E, s E S 
denote the state of link k obtained from ilg for probe i. 
yjf(i) = 1 if the probe passes link k, otherwise Yi^{i) = 0. 
Further, let ils{i) denote the observation obtained by Rs{i) 
from the probes sent by source s. 

C. Sufficient Statistic 

It is assumed that each source sends probes independently, 
and the arrivals of probes at a node or a receiver are also 
assumed independent. Then, the loss process of a link is 
considered an i.i.d. process. This also makes the collective 
impacts of probes sent by the sources to a link i.i.d.. Thus, 
the likelihood function of an experiment takes either a product 
form of the individual likelihood functions or a summation 
form of the individual log-likelihood functions. 

Using the internal view proposed in the tree topology, the 
confirmed passes of link i for the probes sent by source s can 
be obtained from fig that is equal to 



Mis,l)=Y,Yfij). 



In addition, let 



ni{s,0) ^ np(s^i)(s, 1) - ni{s, 1) 

be the number of probes sent by source s that become 
uncertain in T{i). Further, considering all sources, we have 

ni{l) ^ X! '^^(^'1) "-*(0) = ^ nj(s, 0) 

ses{i) seS(i) 



be the total number of probes confirmed from ft that pass 
link i and the total number of probes that turn to uncertain in 
T{i), respectively. Using the same method as that presented in 
section ITlI-DI we can prove rii(l), i e _E is a set of sufficient 
statistics. 

V. Maximum Likelihood Estimator 

Given the similarity between the statistics presented in 
the last section and that presented in section III-CI a link- 
based estimator can be obtained by a similar procedure as 
that presented in section IIII-BI 

A. Link-based Estimator 

Using the set of sufficient statistics, we can write the log- 
likelihood function of il, and using the same strategy as 1231 . 
we have the log-likelihood function of a general network as 
follows: 



L{e) - ^ [n,(l)-?og(l-0,)+"»(O)-^O5Cz 



(12) 



where = 6*,; + (1 - 6',)(1 - /?,)). Differentiating L{0) wrt. 
9i and setting the derivatives to 0, we have a set of equations 
as follows: 



n.,{S(i)A) 



1 - 



ni{S{i), 1) 



nf(^,^{S{i),l) + irnp{S{i)J{i)) .^„„„^ 

i ,Z fc jrSixLi, 



1) 



(13) 



_ EjgS(.o['^/(»)(i. 1) + imp{j, f{i))] 

i e SSNL, 

Ejgs(^) IMj, 0) + impjj, f{i))] . ^ 

Eje5(i)["/wO''l) /(*))]' 

where RL denotes root links, SBRL denotes the links that are 
not root link but only receive probes from a single source, 
SSNL denotes the links in a intersection but leaf ones, and 
AOL denotes all others, i.e. leaf links. As dU, inip{j, i) 
denotes the impact of nk{j, 0), k £ a(j, i), on the loss rate of 
link i. 

imp{j, i) = 

nk{j, 0) • pai{k) ■ * Uieaii) DqecAi ^i 

E - 

fcea(j,i) 



l>k 



Ok + {l-ek)U,ecJi 



where 



pa^ik)= U (1 



lea{j,i) 
l>k 



In fact, the second equation of ( fTST i can be merged into 
the third one since each link in SBRL has its \S{i)\ = 1. 
To distinguish SBRL from SSNL here is for later to divide a 
general network into a number of independent trees at j, j £ J. 
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Fig. 2. A Network with Multiple Sources 

The estimate obtained from ( fT3] l can be proved to be the 
MLE by using the same procedure as that presented in section 
IIII-DI In addition, almost all of the results obtained in the tree 
topology can be extended to the general topology. For instance, 
let 7i(a(i)) be the pass rate of T{i), the empirical pass rate 
of -fi{a{i)) is 

In addition, if Pi can be expressed as a function of Oi, 
the likelihood equation is expressed by a polynomial as (|9|l. 
However, there is a lack of methods to express Pi for the 
general topology because there can have a number of sources 
sending probes to subtree i. 

B. Insight and Remark 

Despite it is unclear how to express Pi, the similarity 
between the four equations presented in (fTji and the three 
in dUi provides such a hope that a path-based estimator as 
([Tol l is available to the general topology. To make this happen, 
equations (|4]l, (fTSl l and ( fTOl l are examined and an insight is 
emerged that is presented in the following remark: 

Remark: regardless of the topology and the number of 
sources, the MLE of the loss rate of a link can be obtained if 
we know: 

1) the total number of probes reaching the parent node of 
the link, e.g. n*, 1) + imp{S{i), f{i)), or 

I^jesw["/w(i'l) /(«))] of (dUi; 

2) the total number of probes reaching the receivers via the 
link, e.g. nj(S'(i), 1), or EjeS(j) 1) °f ^"'l' 

3) the pass rate of the subtree rooted at the child node of 
the link, e.g. P^ of ([T3]l. 

Note that the link stated in the remark can be a path consisting 

of a number of links serially connected. For the tree topology, 

there is only one source, 1) and 2) can be suppressed to the 

pass rate of the path connecting the source node to an internal 

node. ([Tol l as an example takes advantage of this to express 

1 — Pi, the loss rate of subtree i, in two different functions of 

Ai'. a) the product of the loss rates of the multicast subtrees 

rooted at node i; and b) 1 — since AiPi = 7^. 

Ai 

C. Path-based Estimator 

As stated, there has been a lack of likelihood function 
proposed for the general topology regardless of whether it is a 



link-based or a path-based. This is partially due to the lack of 
sufficient statistics of a link or a path from observations. With 
the help of the minimal sufficient statistics ni{l),i e V, a 
link-based estimator has been presented in the paper Further, 
using the remark presented above and the statistics, we are able 
to have a path-based log-likelihood function for the general 
topology. Among the three factors listed in the remark, two of 
them are available, i.e.: 

> the total number of probes sent by a source; and 

> the total number of probes reaching Rs{i) and know the 
source of each arrived probe. 

Given two of the three, a likelihood equation with the third 
factor as the variable can be derived. Let 

A{s,i)= Yl i eV, cind s e S{i) (15) 

jea{s,i) 

be the pass rate of the path from source s to node i. It is 
easy to prove that ( fTsT i is the bijection, F, from 9 to A, where 
Q is the support space of {di,i G E} and A is the support 
space of {A{s, V\S,s £ S{i)}. To have a polynomial 

likelihood equation, we need to express Pi by A{s,i). 

In contrast to the tree topology, there are intersections in 
the general topology. For intersection i. Pi must be consistent 
for all paths that connect a sources to node i. To prove that 
is a necessary condition for the MLE, we have the following 
theorem. 

Theorem 3. The likelihood equation describing the pass 
rate of a path connecting source s to node i, i £ J can be 
expressed as 

A{s,i)^^, s&S{i). (16) 

Pi 

As the tree topology, the empirical probability of ji{s) can be 
obtained by — ^. 

Proof: Using the sufficient statistics, we can write the 
path-based likelihood function as 

P{A{s,i)) = n n [Ms,ir'^'-'H'^-A{s,i) + 

= n n [Ms,^r''''^x 

The log-likelihood function takes the following form: 

L(P(A(s,i))= J2 E [n^{s, 1) log A{s,i) 
iev\s seS(i) 

+ (71,(1) - 7i,(s, 1)) log(l - A{s, i)P^) . 

Differentiating the log-likelihood function wrt A{s,i) and 
letting the derivative be 0, we have 
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A{s,i) 



s e S{i) 



( fTSb is almost identical to that obtained for the tree topology 
except that the Pi is consistent to all 7i(s),s G S{i). We 
call this the consistent condition since it ensures the estimates 
obtained for A{s,j),j G a*(i),s e S{i) consistent with each 
other The condition is also implicitly presented in (fTsl i that 
indicates there is an order in the estimation of a general 
network, where /3i,i E J should be estimated before A{s, k), 
k e a^ii), s e S{i). 

To have the MLE A{s,i),s G S{i),i G J satisfying the 
consistent condition, we have the following two theorems. 

Theorem 4. Let A{k,i) be the pass rate of the path 
connecting k,k £ S{i) to i,i E J in a network of the general 
topology. There is a polynomial, H(A(k,i), S{i)), as follows 
to express the estimate of A{k,i). 



H{A{k,i),S{t))^l 



A{k,i) 

rj^,_ji^^}^fimi^ih}L) = o (17) 



where 



ni{k, 1) 



Proof: Assume source k,k E S{i) sending probes to node 
i. Based on the first equation of (fTST i. we have 



A(fc,z)A-^^i%^=7,(fc), kES{i) 

Since \S{i)\ > 1, there are more than one sources sending 
probes to node i, the pass rates of two sources, s and k,s,k E 
S{i), to node i are correlated that can be expressed as 



A{s,^)^A{k,^)^, 



(18) 



Let n*(l) be the total number of probes reaching node i, we 
have 



Then, we have two ways to express 1 — If the observations 
of the subtrees rooted at node i satisfy dHJ, we have 



jedi 

and using ( fT6] l. we have 



A{k,i) 



(19) 



Connecting the two, we have 



1 - 



A{k,i) 



n 



1 



7i('=)-E.eS(i)'^i(^'l) 



ses{i) 



Except for A{k,i), all others, e.g. ji{k),nj{s,l), are either 
known or obtainable from observations. Thus, the above 
equation is a polynomial of A{k,i). Alternatively, using (fT9] l 
to replace /?; from (fT3T l. we have the same result. ■ 
( fTTb generalizes ( fTOl l that considers various paths ending at 
a common node, including those having \S{i)\ > 1. For node 
i having \S{i)\ = 1, (HtJi degrades to dTUll. For \S{i)\ > 1, if 
3 e di, 



(20) 



is the estimate of the pass rate of the link connecting node 
i to node j, where the numerator is the sum of the probes 
sent by S{i) that reach Rs{i) and the denominator is the sum 
of the probes sent by S{i) that reach Rs{j),j E d{i). Both 
are obtainable from fl. The estimate is built on the arithmetic 
mean that considers the contribution of all sources sending 
probes to the link. 

Solving ( fTTI l, we have A{k, i). Then, we can have 
A{s,i),sE S{i) \ k from ([T3 and 



A{s,i)' 



,seS. 



Using Lemma 1 in 1 9 1, we are able to prove there is only one 
solution to ( fTTI l in (0, 1)™, To prove the estimate obtained 
from (Til is the MLE, we resort on a well known theorem 
for the MLE of a likelihood function yielding the exponential 
family. 

Theorem 5. If a likelihood function belongs to a standard 
exponential family with A{s,i) as the natural parameters, we 
have the following results: 

1) the likelihood equation — has at most one 
solution 9* E O; 

2) if 9* exists, 9* is the MLE. 

Proof: This theorem can be found from a classic book 
focusing on exponential families, such as |24|. The likelihood 
function presented in JTtI i belongs to the exponential family, 
where A{s,i) is the natural parameters. Then, the estimate 
obtained from (TT\ is unique in its support space and it is the 
MLE of yl(s,i). ■ 

VI. Solutions 

Theorem |4] is not only valid to estimate the pass rates of 
a path connecting a source to a joint node, but also valid to 
estimate the pass rate of a path connecting a source to a node in 
an intersection. Then, the loss rate of a link in an intersection 
can be obtained easily since ( fTSl l is the bijection function F 
from to A, and then F'^^', from A to 9, is as follows 



E.gsM n'A{s,i) 



(21) 
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Unfortunately, F^^^^ cannot be applied to the links falling 
into SBRL and having at least a descendant connecting an 
intersection, i.e. the ancestors of an intersection, because of 
the consistent condition. 

A. Consistency and Decomposition 

To estimate the loss rate of link j, j e a{i) /\i ^ J, called 
ancestor links, a divide-and-conquer strategy is proposed here 
that is not only an algorithm to estimate the loss rates of 
a general network, but also a necessary measure to ensure 
the consistency established in theorem [3] Instead of directly 
finding a likelihood equation for the ancestor links from the 
statistics proposed in Section ITV-CI a general network covered 
by a number of trees is divided into a number of independent 
ones before being estimated. The strategy is built on equation 
(fTTI l that shows the parameter estimation in a hierarchical 
structure with probes flowing in one direction can be divided 
into a number of sections as the d-separation stated in [25 1. 

Given the number of probes reaching a node, the descen- 
dants of the node not only become independent from each 
other, but also become independent from their ancestors. Since 
each of the independent parts is a tree, the estimators proposed 
for the tree topology can be used to estimate the loss rates of 
the independent parts. Then, the immediate question is which 
nodes should be selected as the decomposing points that can 
minimize the cost of estimation. Two criteria are proposed 
here for the optimal decomposition, which should: 

1) ensure that all the nodes and links in an independent 
tree receive probes from the same set of sources. Then 
the sources can be considered a virtual source sending 
probes to the receivers of the independent tree; and 

2) minimize the number of independent trees created after 
decomposition since the computation cost is propor- 
tional to the number of the independent trees. 

The following theorem is presented for the optimal strategy to 
select decomposing points. 

Theorem 6. Given a general network covered by multiple 
trees, the optimal strategy to decompose the overlapped mul- 
tiple trees into a number of independent trees is to use the 
nodes of J as decomposing points. 

Proof: Obviously the nodes or links of an intersection 
can only receive probes from the same set of sources. The 
minimum number of independent trees is proved by con- 
tradiction. Firstly, we assume there are two strategies, say 
A and B. Strategy A decomposes a general network into a 
number of independent subtrees and there is at least one of 
the decomposing points that is not a joint point and the total 
number of the independent subtrees obtained is m! . On the 
other hand, strategy B decomposes the network at the joint 
points only and the total number of independent subtrees is m. 
If strategy A is better than strategy B, we should have m' < m. 
However, this is impossible: given the fact that all of the 
subtrees rooted at a joint point can only become independent 
from each other if we know the state of the joint point 
according to d-separation. Then, we have m! > m + 1, which 
contradicts to the assumption. Then, the theorem follows. ■ 



To start the the divide-and-conquer strategy, we need to 
have a method to estimate i € J. If i has more than 6 

descendants, there is not a closed form solution according to 
Galois theory. Then, an approximation method, such as (flTt or 
the EM algorithm, is needed to find the solution. Apart from 
them, an iterative procedure based on fixed point theory can 
also accomplish this. The iterative procedure uses: 



(1 - /3,)(*+i) = 



)(0) , Efee50-)-'(l) 



'-Z]fcGS(i) 



nt{l) E 



n1{l) 



(1 - A)^^^ 

(22) 

to compute (1 — [26|. According to Lemma 1 in ||9l, 
there is only one (1 — in (0, 1) satisfying ( fTTI l. Then, the 
iterative procedure is assured to coverage at {1 — jSi)* and 
the number of iterations depends on the initial value of (1 — 
(1 — If the initial value is selected properly, i.e. 

close to the fixed point, only a few of iterations are needed. 
A number of methods can be used to obtain the initial value, 
one of them is to use a weighted average obtained from the 
estimates of the trees involving in subtree i; another is to use 
the top down algorithm proposed in 1 10|. In our experience for 
a network as Figure |2] the procedure needs 5.25 iterations on 
average to converge to the fixed point when e is set to 10^^*^. 
Alternatively, Newton Raphson algorithm can be applied to 
(I22I) to find the (1-/3^)* that can converge to (1— quickly 
as wefl. Given (1 - Pi)*, we have (3* = !-(!- and 

As stated, the independent trees obtained from decomposi- 
tion can be divided into two groups on the basis of ingress or 
egress of the links connecting or connected to the decomposing 
points. The group consisting of the trees that have at least an 
ingress link to a joint node is called the ancestor group, all 
others that have an egress link from a joint node is called 
the descendant group. For the descendant group, given n*{l), 
the maximum likelihood estimators developed for the tree 
topology can be applied to estimate the loss rates of the 
independent trees. 

Although the trees in the ancestor group as those in the 
descendant group are independent from each other given 
n*{\), they cannot be estimated directly by an estimator 
developed for the tree topology. As previously stated, the pass 
rate of a path ending at a node that is an ancestor of a joint 
node depends on the estimate obtained for the intersection. 
Then, the pass rate of the path connecting source r, r e S{i) 



to node i,i E J should be 



If j is the last link of the 



path, i.e the link ending at node i, the statistic of link j, nj{l), 
is equal to 



n^l) 



,j e p{i). 



(23) 



where p{i) denotes the parents of node i. To ensure the 
consistency of estimation with the pass rates from k to 
a'^{l),l e p{i),k e S{i), we need to use instead 
of Then, (|9]l is no longer valid to the estimation of 



10 



the links of the ancestor group and we need to use the first 
equation of (|4|i in the subsequent estimation. As previous 
stated, we need to consider how to express /3g,q G a^U) to 
have a valid likelihood equation according to riq(s), s £ S{i). 
Under the prefect assumption of f^g(s), s S S{i) fSOl, we use 



dUi can be written as 



A{s,i) 



n|(l) 



to replace 7i(s) in the modified path-based 



estimator that has the following form: 



1 - 



= 1- 



A{s,i) 



n (1 ''^'^ 



)■ (24) 



Note that (|24] | is obtained under the basis of ([8]l, that is a 
polynomial of A(s, /"(i)). Once having yl(s, we are 
able to estimate n*j:^^.~^{l), then we move a level up toward s 
and use ( |24] | to estimate A(s,/|(i)). This process continues 
from bottom up until reaching s. If the tree being estimated 
has more than one intersections, at the common ancestors of 
the intersections, the RHS of (l24l i will have a number of the 
left-most terms, one for an intersection plus the product term 
for those subtrees that do not intersect with others. If the 
total number of intersections plus the number of independent 
subtrees is larger than 5, there is no a closed form solution to 
(El. 

Compared to the tree topology, the general topology, despite 
having a similar likelihood equation, has its unique features in 
estimation and one cannot simply use the estimators proposed 
for the tree topology in the estimation of the general topology. 

VII. Statistical Property of the Estimators 

Apart from providing solution to the general topology, 
we study the statistical properties of the solution, such as 
whether the estimators are minimum-variance unbiased es- 
timators (MVUE); and/or the estimate obtained is the best 
asymptotically normal estimates (BANE), etc. This section 
provides the results obtained from the study. 

A. Minimum-Variance Unbiased Estimator 

The estimators proposed in this paper are MVUE and the 
following theorem proves this. 

Theorem 7. The estimator proposed in this paper is MVUE 
and the variances of the estimates reach the Carmer-Rao 
bound. 

Proof: The proof is based on Rao-Blackwell Theorem that 
states that if g(X) is any kind of estimator of a parameter 9, 
then the conditional expectation of g(X) given T(X), where T 
is a sufficient statistics, is typically a better estimator of 6, and 
is never worse. Further, if the estimator is the only unbiased 
estimator, then, the estimator is the MVUE. 

To prove the estimator is an unbiased estimator, we use (|4) 
instead of (fTol l since the former is not only more general than 
the latter, but also much simpler than the latter in the proof. 



1 - ^ 

Let = 1 - 6'i. We then have 
n,(l) 



(25) 



E 



= E\ 



Pi 



E 



'-fb 



)(1) 



E{e,) 



= E 



- 0,: = 



j=i 



(26) 



The statistics used in this paper has been proved to be the 
minimal complete sufficient statistics. Then, applying Rao- 
Blackwell theorem, the theorem follows. 

Given theorem [T] it is easy to prove the variance of the 
estimates, e.g. Ot and /3i, obtained by ( [TtT i are equal to Carmer- 
Rao low bound since ( fTTl ), the likelihood function, belongs to 
the standard exponential family. ■ 

Based on Fisher information we can prove the variance 
of the estimates obtained by ( fTTb from Vl is also smaller 
than that of an estimate obtained by ( fTOl i from Vts^s G S. 
Since the receivers attached to intersections observe the probes 
sent by multiple sources, the sum of the observed probes 
is at least larger than or equal to the maximum number of 
probes observed from a single source. Therefore, there is more 
information about the loss rates of the links located in the 
intersections since information is addictive under i.i.d. assump- 
tion. With more information, the variance of an estimate of 
a parameter must be smaller than another obtained from the 
probes sent by a single source according to Fisher information. 
Given the less varied estimates from the intersections, the 
variances of the estimates of other links that are not in an 
intersection are also reduced, at least not increased. Therefore, 
the estimates obtained by ( fTTj l is better than those obtained 
from a single source, which also implies the necessity of an 
order in the estimation of a general network. 

B. General Topology vs. Tree Topology 

The results presented in this paper can be viewed as a 
generalization of the results presented in [9| since the tree 
topology is only a special case of the general topology. The 
findings and discovery presented in this paper cover those 
presented in |9|. The following corollary confirms this: 

Corollary 1. Any discovery, including theorems and algo- 
rithms, for loss estimation in the general topology, holds for 
the tree topology as well. 
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For instance, ( fTTj l obtained for the general topology holds 
for the tree one. When S{i) — {k}, we have 

7.(fc) 



H{A{s,k),S{k)) = 1 



A{k,i) 



-pr . 7»(fc)EsgS(»)"3(^^l) . 

_ 7»(fc) _ TT ^1 7»(fc)"3(l) X 



A{k,i) 



7,(fc) 







(27) 



the last equation is H{Ai, i) presented in [lO). 

On the other hand, we are also interested in whether those 
properties discovered from the tree topology can be extended 
into the general one and the difference between the original 
properties and the extended ones, in particular for the rates of 
convergence. 

C. Large Sample Behavior of the Estimator 

Since the estimate obtained by the proposed estimator is 
the MLE 9i, we are able to apply some general results on 
the asymptotic properties of MLEs in order to show that 
\fn{6i — 6i) is asymptotically normally distributed as n — > oo. 
Using this, we can estimate the number of probes required to 
have an estimate with a given accuracy for many applications. 
The fundamental object controlling convergence rates of the 
MLE is the Fisher Information Matrix at Oi. Since L{6) yields 
exponential family, it is straightforward to verify that 6i is 
consistent and that L(6) satisfies conditions under which / is 
equal to 



(0) 



Eliminate singular on the boundary of (0, 1]'^', we have 

Theorem 8. When 6, e (0,1), i £ V \ S,y/n{§i - 9,) 
converges in distribution as n ^ oo to an \E\ dimensional 
Gaussian random variable with mean and covariance matrix 
I-H9), i.e. 

V^{9-9) ^ N{0,I-\9j) 

and 9i is the best asymptotically normal estimate (BANE). 

Proof: It is know that under following regularity condi- 
tions: 

• the first and second derivatives of the log-likelihood 
function must be defined. 

• the Fisher information matrix must not be zero, and must 
be continuous as a function of the parameter 

> the maximum likelihood estimator is consistent, 
the MLE has the characteristics of asymptotically optimal, i.e., 
asymptotically unbiased, asymptotically efficient, and asymp- 
totically normal. The characteristics are also called BANE. 

It is clear that ([TtI i. the likelihood function used in this 
paper, belongs to the standard exponential family, which 
ensures the consistence and uniqueness of the MLE. To satisfy 



the second condition for the exponential family, ni{s,\),i £ 
E,s E S should not be linearly related, this is true as 
ni{s,l),i e E,s e S have been proved to be the minimal 
sufficient statistics. Then, we only need to deal with the first 
condition. Obviously, ( fT3T l has both first and second derivatives 
in (0,1)1^1, and L{9) is strictly concave, which ensures the 
Fisher information matrix I{9) positive definiteness. ■ 
Theorem |8] states such a fact that with the increase of the 
number of probes sent from sources, there are more probes 
reaching the links of interest. Then, there is more information 
for the paths to be estimated. Let Iq{9) is the Fisher Informa- 
tion for a single observation, we have I (9) — nIo{9), where 
n is the number of observations related to the link/path being 
estimated. 

As n — > oo, the difference in terms of Fisher information 
between the two estimators approaches to zero. Therefore, 
as n — > oo, estimation can be carried out on the basis 
of individual tree and the asymptotical properties obtained 
previously for the tree topology Q hold for the general 
topology as well. On the other hand, if n < oo, the estimate 
obtained by ( fTST i or (fTTT i is more accurate than those obtained 
from an individual tree. The simulation results presented in the 
next section illustrate this that shows the fast convergence of 
the estimates for the links located in the intersection because 
there are more information about the links. 

Although there are a number of large sample properties 
that can be related to loss inference in the general topology, 
including various asymptotic properties, we would not discuss 
them further because n — oo means a large number of probes 
be sent from sources to receivers that requires a long period of 
stationarity of the network, including traffic and connectivity, 
which is impractical based on the measurement II27I . 

VIII. Simulation Study 

For the purpose of proof of concept, a series of simulations 
were conducted on a simulation environment built on ns2, the 
network simulator 2 ll28l . The network topology used in the 
simulations is shown in Figure |2] where two sources located at 
node and node 16 multicast probes to the receivers attached 
to the leaf nodes. Both use a constant rate to send probes to the 
receivers, where the interval between 2 probes is set to 0.01 
second. The binary structure used here is for the simplicity 
reasons. Apart from the traffic created by probing, a number of 
TCP flows with various window sizes and a number of on/off 
UDP flows with various burst rates and on/off periods are 
added at the roots and internal nodes that acting as background 
traffic send packets to the receivers attached to the leaf nodes. 
Each simulation run for 200 times and each time a random 
seed is selected to start the simulation. The samples collected 
in a run vary from 200 to 2000, with an interval of 200, to 
measure the impact of the sample sizes on the accuracy of 
estimation. The accuracy of estimation is measured by the 
relative error that is defined as: 

afos(actual loss rate - estimated loss rate) 
actual loss rate 

To compare the estimates obtained from end-to-end observa- 
tion with those obtained directly at each node, an agent is 
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added at each node to record the number of packets lost and 
passed a Hnk. The ratio between the losses of a link and the 
sum of the losses and passes of the link is used as the actual 
loss rate of the link. As stated, the impact of background traffic 
on the accuracy of estimation is also a concern of this study. 
The next two subsections are devoted to these two issues. 

A. Impact of Sample Size on Accuracy 

In the first round simulation, there are 9 TCP flows in the 
multicast tree rooted at node 0, where 

• 4 from node to nodes 8 and 9; 

• 2 from node to nodes 10 and 11; and 

• 3 from node 2 to nodes 9, 10 and 11. 

In addition, there 12 UDP flows in the multicast tree, including 

• 2 flows from node to node 8 and node 9; 

• 4 flows from node 3 to nodes 12-15; 

• 2 flows from node 2 to nodes 10 and 11; 

• 2 flows from node 1 to nodes 10 and 11; and 

• 2 flows from node 4 to nodes 8 and 9, respectively. 
The window size used by the TCP flows is 50. 

For the multicast tree rooted at node 16, there are 6 TCP 
flows and 4 UDP ones. Among the 6 TCP flows, 2 of them 
are from node 17 to nodes 8 and 9, the other 4 are from node 
16 to nodes 21-24, respectively. The UDP flows are from node 
18 to nodes 21-24. The window size of the flows is set to 60. 
The loss rate of each link is controlled by a random process 
that has 1% drop rate. 

The resultant relative errors of the 24 links are presented 
in Figure |3] to Figure |5] 8 links in each. The figures show that 
with the increase of samples, the relative errors are decreased 
as expected. The relative errors of links to 15 drop to 
around 10% when 2000 samples are used in estimation. This 
phenomenon is consistent with the expectation, i.e. the number 
of samples used in estimation is inversely proportional to 
the variance of an estimate. However, whether we are able 
to use more samples to achieve the required accuracy is a 
question that has not been answered previously. What we 
found from the simulation is that the loss process of a link 
is not independent from background traffic. Then, we need to 
investigate traffic dependent models in the future. 

B. Impact of Background Traffic on Accuracy 

Comparing Figure |3] with Figure |5] one is able to notice the 
difference between them, where the relative error of Figure |5] 
is significantly larger than that of Figure |3] Another interesting 
point between the two figures is the improvement rate of 
the accuracy against the number of samples, where the rate 
appears gradually reduced as the sample sizes in Figure |5] 
When the sample size reaches 1600, further increasing the 
number of samples only has a marginal impact on the accuracy. 
This phenomenon triggers us to consider the causes of the 
reduction since the network structure used in the simulation is 
symmetric and the parameters used for each link is identical. 
Apart from those, both sources use the same rate to send 
probes to the receivers. The only difference between them 
is the background traffics flowing on the links. The traffic 



flowing on the subtrees rooted at node 3 consists of a number 
of on/off UDP flows. In contrast, there are four FTP flows 
sending packets from node 16 to nodes 21, 22, 23, and 24, 
respectively, plus four on/off UDP flows forwarding traffic to 
the 4 receivers from node 18. Thus, there are two possible 
causes that lead to the difference: 

• a mismatch between the traffic patterns of the probe flow 
and the background flows, and 

• a mismatch between the traffic intensities of the two type 
of flows. 

It is hard to overcome the former if the loss model of a link is 
related to traffic unless knowing the correspondence between 
models and traffic in advance and having an estimator for each 
model. Without them, increasing the number of samples is the 
only option that may improve the accuracy in some degree, 
but it also has a side-effect as previous stated. To have more 
probes for estimation requires the traffic remains stable. If the 
duration of stability is also an issue, what we can do is to 
increase the intensity of probing, i.e. sending more probes in 
a period. However, the probing may intervene the background 
traffic that can result in an incorrect estimate. 

To confirm the above, two rounds of simulations are carried 
out: both stem from the previous setting, one removes the 4 
TCP flows from node 16 and the other removes two of the 4 
TCP flows. The result of the former is presented in Figure 
that clearly shows the improvement in terms of accuracy. The 
relative error remains steady at around 5% for links 18-24, 
irrespective to sample sizes. This is because of the consistence 
between the models used by the estimator and the background 
traffic. In contrast, when 2 of the 4 TCP flows are restored, one 
sends probes from node 16 to node 21 and the other to node 23, 
the relative errors of the links having TCP traffic are bounced 
back to where they were, as shown in Figure |7] However, the 
relative errors of link 22 and link 24 remain at the same level 
as those in Figure |6] Further, we halve the window size of 
the TCP flows to see its impact on the accuracy. The result 
is presented in Figure H] Comparing Figure |8] with Figure 
there is little difference between them. This reflects the loss 
process created by TCP flows are similar to each other that is 
independent from the number of flows and size of congestion 
window. 

The results presented in this subsection indicate that apart 
from the sample size, to have an accurate estimate we must 
consider the correspondence between the model used in es- 
timation and that of traffic. Without the knowledge, we can 
only adjust the ratio between the probes sent by a source and 
that of the ongoing packets to improve estimation accuracy. 
For instance, in the first round, the background traffic for the 
shared subtree is much higher than other links and with both 
TCP and UDP. Nevertheless, the relative errors of the links in 
the intersection can reach about 10% when 4000 samples are 
used in estimation that are significantly better than those of 
links 18-24 since the sending rate of probes for those links is 
two times of others. 

IX. Conclusion 

In this paper, the findings obtained recently on loss to- 
mography are presented that include theoretical findings and 
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practical algorithms. In theory, the internal view introduced 
in the paper leads to a set of complete minimal sufficient 
statistics that contain all the information needed to compute the 
loss rates of a network. Based on the statistics, the frequently 
used likelihood function is rewritten, and subsequently a set 
of likelihood equations is obtained. Solving the likelihood 
equations, a direct expression of the MLE is obtained for 
the link-level loss rates of the tree topology. In contrast 
to the previous works, the direct expression considers the 
dependency between a likelihood equation and the data set 
obtained from experiment. Because of the dependency, the 
direct expression proposed in this paper is applicable to all 



data sets, while the previous ones is at most applicable to one 
type of data set. 

With the success achieved on the tree topology, two direct 
expressions of the MLE are derived for the general topology, 
one is a link-based estimator and the other is a path-based 
one. The former has a similar structure as the one derived for 
the tree topology, which also ensures most of the theorems 
obtained for the tree topology hold for the general topology. 
The latter is a polynomial that has a similar structure as (|T0] ). 
In fact, the latter generalizes the path-based estimator proposed 
for the tree topology. In addition to the likelihood equations. 
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the dependency between estimates has been identified in the 
estimation of the general topology that subsequently leads to 
the introduction of estimation order for the general topology. 
To impose the order in estimation, a divide-and-conquer strat- 
egy is proposed that decomposes the trees used to cover a 
general network into a number of independent trees. Apart 
from the divide-and-conquer strategy, a number of methods are 
proposed to estimate the number of probes reaching the roots 
of the independent trees, including a fixed-point procedure. 
Further, the independent trees obtained from decomposition 
are divided into two groups called descendant and ancestor, 
respectively. The estimators developed for the tree topology 



can be used on the descendant group, while a new estimator 
is proposed to handle those falling into the ancestor group. 
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